1. Technical Field
This disclosure relates to image enhancement technology.
2. Description of Related Art
Billions of digital images are taken every day. These are taken using devices such as handheld cameras and phones and video cameras run continuously in cities, buildings, factories, and cars. Trillions of images are already posted on the web for myriad business and personal uses. Given this pervasive and growing role of digital still and video images, technologies that can automatically boost the clarity and visual impact of main objects in these images, both on screen and on paper, can be of great economic significance. However, tools that automatically enhance clarity or salience of images may not target main objects, and may not take best advantage of the cues that are known from perceptual studies to increase apparent contrast between objects and their surrounds.
Human perception of natural objects is primarily based on contours, including occluding boundaries (where an object ends and a background begins), abrupt changes in surface orientation (such as where two faces of a cube meet), and major surface markings (such as the stripes on a zebra). Artists have long known of the importance of contours for visual perception, and therefore use contours to represent objects and scenes in pen-and-ink line drawings. Line drawings involve only a tiny fraction of the original image “pixels,” but often contain all of the information needed to rapidly and reliably recognize objects and scenes. However, efforts to enhance images in a way that selectively and flexibly boost main object contours can provide less than optimal results.
One approach to boosting images salience is to increase overall contrast. However, every pixel in the image may be affected. This may lead to marked changes in image appearance. Shadows may be deepened, bright areas may be washed out, and colors may become unnaturally saturated. At the same time, objects may not be emphasized.
Another approach is to selectively boost mid or high spatial frequency bands, as in “unsharp masking” and other sharpening approaches. See Leat, S. J., Omoruyi, G., Kennedy, A., and Jernigan, E. (1005), Generic and customized digital image enhancement filters for the visually impaired, Vision Research, Vol. 45, No. 15, pages 1991-2007. doi:10.1016/j. visres. 2005.01.028. This approach may be used to boost details, rather than main objects, to counteract blur caused by low quality lenses or loss of resolution during printing. Concerns with sharpening methods based on spatial frequency filtering may include the fixed relationship between the spatial scale and form of the local image structures that have been targeted by the filtering stage, and the spatial scale and form of the image enhancement itself. For example, when sharp localized structures are targeted in the filtering stage, image enhancement may also be sharp and localized. This may not lead to a desired perceptual effect. Similarly, there may be a simple, monotonic relationship in sharpening between the strength of the image feature and the strength of the image enhancement: stronger image features receive greater enhancement. This may be the opposite of what is wanted: strong edges may not need enhancement but receive it anyway, and weak edges—for example, places along an object boundary where the object blends into the background and local contrast is lost—may need the most enhancement but receive none.
Another approach to enhancing “objects” is the use of an edge detection algorithm to select image locations to boost. Edges may be the elements of object contours. This technique has been used in “smart sharpening” algorithms. But once edges have been identified, the problem may remain that the spatial scale and magnitude of the enhancement is tightly coupled to the spatial scale and magnitude of the underlying image structures.
Once edges have been identified, an alternative use of edges is to superimpose lines on the original image where edges have been located. These lines can be either black, as in a traditional line drawing, or “bipolar” in which both black and white lines are superimposed in pairs on the light and dark sides of the edge. See Peli, E. S. & Peli, T (1984), Image enhancement for the visually impaired. Optical engineering, Vol. 23, No. 1, pages 047-051. While the width of the superimposed lines can be varied in such an algorithm according to the choice of the user, the original image pixels are replaced by the enhancement. This may lead to loss of information and can produce an unnatural, cartoon-like appearance.
Another concern that can occur in both sharpening and superimposed line-based approaches is that, as the enhancement level is turned up and begins to measurably increase the salience of the targeted structures, the processed images may begin to take on an undesirable appearance that some viewers may find cluttered or harsh.
The Razor Vision™ Video enhancement cable by Belkin (˜$200), follows the above-mentioned filtering-based approach. However, it indiscriminately boosts contrast within a specific spatial frequency range. This may highlight uninformative background textures that can actually impede vision.